Reviewing
Requirements Determination in Information Systems Projects
Information Systems 6840
University of Missouri-St. Louis
20 November 2015
Luke Held
Contents
Frederick
Brooks asserts that software systems are inherently complex, and that
complexity must be dealt with and managed, not abstracted away: “The complexity
of software is an essential property, not an accidental one. Hence descriptions
of a software entity that abstract away its complexity often abstract away its
essence.”[1]
According to many researchers, a not insignificant amount of that complexity is
generated early in the development lifecycle of systems projects, during
requirements determination, making it a critical stage in the project
lifecycle.
Requirements
determination has many names and shades, including requirements elicitation,
information requirements discourse[2],
information requirements determination[3],
requirements determination[4]
(and relatedly, service requirements[5]),
requirements engineering, and simply requirements analysis[6].
Although each term includes its own perspective and connotations, I will use requirements
analysis (RA) throughout for simplicity, following Alvarez. Alvarez defines RA
as a process of “eliciting and encoding into the new system the requirements
that clients verbalize to analysts[7].”
Irrespective of the term employed, however, researchers and practitioners agree
that while RA is a critical component of systems development[8],[9],[10],[11],
it is also highly problematic, being an area of endemic miscommunication
between stakeholders and leading to significant problems down the road[12],[13],
including persistent requirements uncertainty and project management issues[14],[15].
A review of some of the various RA methodologies is given next, followed by a
discussion of their commonalities and differences, under which circumstances
they are likely best applied, and ultimately what can be learned from them in
order to better determine requirements in systems projects.
Jiang et. al. examine RA through a lens
of requirements uncertainty, including the various components of that
uncertainty and the perception gaps of the stakeholders involved. Key aspects
of requirements uncertainty in Jiang’s model include: requirements instability
(changes in requirements throughout the project) and requirements diversity
(user disagreement over the requirements)[16].
Other aspects of requirements uncertainty include incomplete or ambiguous
requirements and lack of user support. Via analysis of a survey of more than
150 information systems (IS) professionals, the authors confirm their most of
their hypotheses, including those that requirements instability and diversity
increase the perception gap and subsequent requirements uncertainty.
To mitigate
these issues, Jiang’s team advocates for horizontal coordination between
stakeholder groups, specifically direct communication between users and
developers. This forms part of their broader recommendation that organizations
must work to improve alignment between users and developers both at the outset
and throughout the duration of systems development projects. Specific
techniques include what they term “pre-project partnering,[a]”
which starts before specific tasks are undertaken and where success factors and
measurements are jointly determined.[17]
Organizational change is also recommended to align project goals with
organizational ones, including concomitant changes to organizational and team
cultures. Strong communication structures are also important in Jiang’s
analysis, owing to the different perspectives of the involved stakeholder
groups[b].
They recommend both formal and informal project reviews to facilitate knowledge
sharing.
Brown and
Ramesh similarly note that RA is a crucial component of systems projects, while
observing that it is often pursued in an ad hoc manner and is poorly
understood. However, they also observe that since it occurs early, improvements
at this stage could have outsized benefits for the project overall.[18]
Brown and Ramesh focus on problems encountered during RA and group them into
four classes: 4) constraints on humans as information processes; 2) variety and
complexity of requirements; 3) communication problems; and 4) user
unwillingness to provide requirements.[19]
A variety of factors underlie these issues, including memory limitations (and
how memory interacts with learning[20]),
social and political pressures, instability of user needs/preferences, and differing
perspectives due to different backgrounds.
To address these
problems, Browne and Ramesh posit a number of approaches, including explaining
terminology and discussing potential biases at the outset, with the goal of
limiting them by making all involved parties aware of their impact. Further, they
assert that improving alignment by explaining how sharing information benefits
the user and organization as well as the analyst and the systems project can
reduce motivational biases and political pressures, echoing earlier research.[21]
Use of what-if questions can prompt users to extend their thinking beyond what
is into what could/should be, and introducing surprise[22],[23]
into test tasks can prompt experienced users to retrace their steps in order to
complete an action, thereby rendering explicit what was otherwise unconscious
knowledge. Creation of (after detailed explanation to users to aid
comprehension and alignment) models, diagrams, and charts can make unclear and
abstruse aspects of a system or workflow more salient.[24],[25]
Lastly, as RA is inherently complex and humans are faced with many constraints,
Browne and Ramesh caution that improving RA is an incremental process, building
on prior knowledge and extending it via new projects and approaches to better
address enduring problems.
Alvarez looks
at RA through a dialectical lens (termed critical discourse analysis), a
dialogue between analysts and clients strongly mediated by their different
frames of reference. Similar to others, Alvarez observes that RA is beset by
communication problems and that failures during RA often lead to failures of
systems projects. Although there are many RA methods through which requirements
are determined, Alvarez notes that interviews continue to be a key aspect of RA
and focuses on interviews—and the core aspect of interviews, talk—as she
examines the processes of RA. She goes a step further in asserting that
language and talk underlie RA and systems development by declaring that
“information technology and systems is not about the design of physical things,
it is about the design of practices and possibilities realized through
linguistic communication systems.”[26]
A core
component of those communication systems is framing, a process involving speech,
tone, and expression and through which speakers impart how their messages should
be deciphered. As RA stakeholders, especially users and analysts, come from
different backgrounds and bring different perspectives to the table, framing is
often a contentious process, one wherein power relations are revealed and
sometimes contested. A central area of contention is that analysts view RA in
technical terms, seeking brief answers (often yes or no ones) to questions,
while users employ stories to describe their tasks and needs. Alvarez sums it
up thusly: “for the analyst the encounter is a technological encounter, for the
client it is about personal work experiences and dilemmas.”[27]
Power imbalances between the two groups also exist, in that the analysts
generally control the narrative by asking the questions, deciding what should
be discussed, when, and under what terms.
Alvarez
concludes that the analysts’ technological frames predominant during RA, with
continual but limited counter-framing by users. She further notes that the
imbalanced framing often seen during RA runs counter to key goals of the
process, namely, effective requirements gathering and ultimately effective
systems (which, she observes, will be used by the [aptly named] users rather
than by analysts), and that the involved groups will need to work together to more
fully understand the other’s perspectives in order to create better
requirements and better systems.
Similar to
Alvarez, Halaweh asserts that RA efforts often fail due to overemphasis on
technological frames of reference, and like other researchers cites communication
shortcomings between users and analysts as a central issue. To address these
problems, Halaweh proposes a grounded theory (GT) approach, wherein
preconceived notions are set aside and data are continually defined and refined
through an iterative coding process. (Setting aside predefined concepts is
considered essential in GT, as Halaweh argues that systems often fail because
analysts believe the proposed system is similar to previous ones with which
they are familiar and thus the requirements are already known.) As part of GT,
Halaweh notes that it is often useful to distinguish between user requirements
(services or tasks the system needs to perform) and system requirements
(detailed, technical descriptions of what the system does). Halaweh asserts that
GT helps to identify non-technical requirements through continuous and
iterative engagements with users. (A potential limitation of his research,
which Halaweh explicitly admits, is that GT may skew too far toward
non-technical requirements. In his case study, he notes that few technical
requirements were generated using a GT approach, with the strong majority being
user / task-oriented ones.)
Kirsch
examines RA from the perspective of developing global systems, and the
particular difficulties involved in balancing enterprise and local needs in
systems development. She focuses on two aspects of the RA stage of global
systems projects: knowledge acquisition, described as comparatively structured
and rational (as users understand the need for a global system and the general benefits
of standardization), and negotiation, wherein users vie to have their
perspectives heard and argue for their requirements to become the
requirements used in the new global system.
Kirsch
defines a global or common system as one in which “core software modules [are]
designed around common global requirements to accommodate the needs of multiple
business units within a firm.”[28]
Similar to the other researchers, Kirsch points out that the RA phase is an
inherently difficult one in systems development, citing lack of domain
knowledge as a key issue. Kirsch defines domain knowledge as including both
knowledge of business processes as well as the technical skills required to
build systems, while noting the difficulties in obtaining such composite knowledge.
Again similar to others (especially Alvarez) Kirsch highlights the importance
of user involvement in the design of effective systems. (Further reinforcing
Alvarez’s findings, Kirsch cites another study comparing traditional user/analyst
interviews with a dialectical approach focusing on mutual understanding and
information sharing, wherein the dialectical approach yielded superior
results.) Kirsch’s work involves two cases, and the successes in each are
related to the inclusion of key stakeholders who possess both business and IS
experience (and were perceived to have both by their organizations, thereby
increasing buy-in to and eventual compliance with the final system).
Kirsch’s RA
process includes open coding practices, similar to Halaweh. One benefit of this
approach is to get close to users, generating interesting user-driven benefits.[c]
Another is the acknowledgement of the need to collaborate and compromise, as
when various regions hammered out common terminology to be used, with each
giving up some of their local usage (a sometimes contentious process). The
shared focus also enabled practical prioritization, as when big or high-revenue
regions were given greater sway in determining some requirements (the smaller
regions being somewhat more accepting of this, not just due to recognition of
size differences and business needs, but as they had been involved in
terminology sessions and witnessed their larger counterparts accepting changes
to some of their own preferred terms, implying a sense of fairness through
shared sacrifice). The creation and use of superior shared tools and
infrastructure also fostered acceptance of other tradeoffs.
Despite the
recognition of the benefits of global solutions, Kirsch notes that local buy-in
was often hard to achieve. Frequently users would agree in principle to global
requirements, but continue with their local processes in practice; creating
sufficient acceptance and compliance was difficult, and ensuring users had a
clear understanding how the global requirements would benefit them was key.
Allowing local groups sufficient flexibility to employ customized approaches when
appropriate was also essential. Moreover, learning when to give and take on standardization
versus flexibility was an ongoing effort. Balancing these fundamental tensions
was especially precarious as requirements changed or new ones emerged, as when
new regions were brought into the process, necessitating the reopening some
dearly closed discussions. Skillful facilitators were integral to success, and
were observed to pursue two goals: 1) encouraging participation to ensure all
relevant sides were heard, and 2) focusing requirements and driving for
consensus. This dual focus required continual rebalancing, in addition to great
persistence and perseverance. Having facilitators with both business and IS
experience was also perceived as crucial to achieving needed buy-in to and
compliance with the final systems. Kirsch also notes that involvement in the
negotiation process itself could change perceptions and even desired
requirements, as participants learned new approaches and processes from their
counterparts in other business units. Another takeaway was the technique of
initially categorizing efforts as: new system building, system replacement, or
consolidation of existing systems, aiding in shaping focus in the early stages
of RA.
Duggan
examines RA through the specific techniques of joint application development
(JAD) and nominal group technique (NGT). Key findings include (in a laboratory
setting) that while JAD was found to be relatively effective as a standalone
method, inclusion of NGT led to superior results over JAD alone (specifically,
the integrated approach [referred to as NJAD] was as efficient as JAD by
itself, while reducing the requirement of excellent facilitation skills necessary
with JAD).[29] Similar
to others, Duggan notes the criticality of RA and the communication issues
often encountered at this stage. He also observes that a follow-on effect of
this problem is high maintenance costs for many systems, stemming from the
continuous changes and fixes required due to poor initial requirements.
Duggan describes
NGT as a process wherein 1) individuals generate ideas; 2) the facilitator
records all ideas in a round-robin format; 3) ideas are then discussed for
clarification purposes only, without judgment; 4) ideas are then independently
rated and ranked by participants; and 5) the group prioritizes ideas based on
voting and mathematical assessment of the individual rankings. Duggan also
notes that rather than striving for equal participation, generating
participation according to knowledge and experience is more important. Through his
experiments, Duggan noted that despite its more rigorous structure, NJAD did
not take more time than JAD alone, thereby not increasing the time needed to
complete the RA process; moreover, requirements quality was also found to be
high. Similar to Alvarez, Kirsch, and others, Duggan further observed that
acceptance of and compliance with a system is often as reliant on social
factors as on technical ones, and the mix of individual and group processes in
NJAD was found to produce superior requirements and buy-in compared to JAD
alone.
Many themes and threads emerge from the foregoing discussion. First,
the complexity of RA is unanimous. One aspect of that complexity that is
highlighted by multiple researchers is that requirements are unstable, that
they can change during RA, throughout the project, and even within individual
users, as both circumstances evolve and preferences shift.[30],[31]
Communication problems are another oft-cited issue,[32],[33]
with the most common culprit being different backgrounds and frames of
reference. Also, the observation that RA is ongoing and iterative, formulated
by Halaweh, is supported by others as well.[34],[35],[36]
A key item is not just if various techniques or approaches are
effective, but under which circumstances. While interviews were found to be
useful by multiple researchers (including Alvarez and Kirsch), they do not
address the potential of dominating persons, which may overpower their more
reticent but perhaps no less experienced or expert peers. In those situations,
employment of NGT and similar techniques to generate broader and deeper idea
sets may be warranted. Moreover, Duggan asserts that JAD and NJAD are superior
to interviews in terms of RA efficacy and efficiency.[37]
While likely conceding the efficiency point, Alvarez would nonetheless likely
contest the efficacy one, believing that greater mutual understanding achieved
through an in-depth dialectical approach (although admittedly difficult) more
strongly enhances user buy-in and compliance (and thus overall system
effectiveness). The key components here are time and system complexity. If a
system is comparatively straightforward, JAD (or NJAD) may be sufficient, but
if sufficient uncertainty exists (or buy-in is likely to be especially
difficult), thorough interviews with key stakeholders may be needed as well.
While most of the researchers’ recommendations appear sound, the
practical utility of some findings is questionable. Notably, a key part of
Kirsch’s results involves the participation of individuals with both business
and IS experience (and who are perceived to be knowledgeable about both by
others). While the inclusion of such people in RA is doubtless beneficial,
persons with that combination of experience and skills are likely rare, and
thus relying on them may work in large endeavors where there is sufficient
revenue or risk at play to warrant their input, but it would likely be
difficult in small-to-medium projects, when their duties and responsibilities
may pull them elsewhere.
While differences exist
in the methodologies examined and the conclusions drawn from them, the
similarities are the more salient overall. Notably, specific techniques or
approaches aside, all researchers agree that RA is a complex component of a
complex endeavor, and that communication issues inherent to RA must be dealt
with in order to create effective systems and to ultimately ensure their use.
Working with individual with strong self-awareness and social skills (and to
the extent possible, providing training in these areas to key stakeholders who
lack them) will benefit the RA process, the systems project, and the
organizational overall. Nevertheless, while persons with strong business,
interpersonal, and technical skills are significant assets, they are likely to
be in short supply on most projects. Therefore, while creating and following a
clear approach to systems projects will provide a useful framework, recognizing
the uniqueness of each project is crucial to ensuring the appropriate amount of
flexibility within a given framework is provided to ensure stakeholder buy-in
and ultimately system success.
[a] Jiang’s description of pre-project partnering is similar to a process called early engagement in my own organization. The extent to which early engagement is effectively pursued and implemented varies considerably across project teams in my experience, indicating that more organizational and cultural change is needed to better embed early engagement / pre-project partnering into organizational processes.
[b] The criticality of effective communication is reinforced during a 16 November 2015 lecture by Sharyn Lemmons, PMP. Ms. Lemmons, who has been working in project management for nearly 25 years in the private sector, emphasizes that in her experience strong communication is the single most important component of successful projects.
[c] One such benefit occurred in a
financial services firm studied by Kirsch, where users took the various types
of letters of credit then in use, and wrote specifications on the letters,
creating direct links between their current tasks and proposed requirements,
thereby enhancing the perceived quality of the requirements and increasing
organizational buy-in.
[1] Brooks FP. (1995). The mythical man-month. Reading Mass: Addison Wesley Longman, 183.
[2] Alvarez, R.
(2002). Confessions of an information worker: a critical analysis of
information requirements discourse. Information and Organization, 12, 85–107.
[3] Browne GJ, Ramesh V. (2002). Improving information requirements determination: a cognitive perspective. Information and Management, 39, 625-645.
[4] Kirsch LJ, Haney MH. (2006). Requirements determination for common systems: turning a global vision into a local reality. Journal of Strategic Information Systems, 15, 79-104.
[5] Estebanez LR et. al. (2015). Enhancing service requirements of technical product-service systems. Procedia CIRP, 37, 7-11.
[6] Halawah M. (2012). Using grounded theory as a method for systems requirements analysis. Journal of Information Systems and Technology Management, 9(1), 23-38.
[7] Alvarez, 86.
[8] Yang H, Tang J. (2005). Key user roles on web-based information systems requirements. Industrial Management & Data Systems, 105(5), 577-595.
[9] Business requirements analysis: clearly agreeing what you’re going to deliver. (n.d.) Retrieved from https://www.mindtools.com/pages/article/newPPM_77.htm
[10] Olson JV. (2010). Accurately identifying your requirements—will any computer system be right for you? Journal of Validation Technology, Winter, 29-31.
[11] McMurtrey
M. (2013). A case study of the application of the systems development life
cycle (SDLC) in 21st century health care: something old, something new?
Retrieved from http://quod.lib.umich.edu/j/jsais/11880084.0001.103/--case-study-of-the-application-of-the-systems-development?rgn=main;view=fulltext
[12] Browne and Ramesh, 625.
[13] Alvarez, 86.
[14] Jiang J, Klein G, Wu S PJ, and Liang TP. (2009). The relation of requirements uncertainty and stakeholder perception gaps to project management performance. The Journal of Systems and Software, 82, 801-808.
[15] Sarigiannidis L, Chatzoglou PD. (2011). Software development project risk management: A new conceptual framework. Journal of Software Engineering and Applications, 4, 293-305.
[16] Jiang, 802.
[17] Ibid, 807.
[18] Browne, 625.
[19] Ibid, 627.
[20] Baer
D. (2015, October 27). 4 strategies for remembering everything you learn.
Retrieved from https://agenda.weforum.org/2015/10/4-strategies-for-remembering-everything-you-learn/?utm_content=buffer24222&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
[21] Davis FD. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, September, 319-340.
[22] Surprise
and learning. (n.d.) Retrieved from http://changingminds.org/explanations/learning/surprise_learning.htm
[23] NSF
funds probe of the quintessence of surprise. (2005, November 28). Retrieved
from http://www.eurekalert.org/pub_releases/2005-11/uosc-nfp112805.php
[24] Sharma
A. (2015, April 28). Data flow diagrams—are they worth it? Retrieved from http://www.batimes.com/articles/data-flow-diagrams-are-they-worth-it.html
[25] Maxted
SJ. (2008, December 1). The business context model: as good as it gets.
Retrieved from http://www.batimes.com/articles/the-business-context-model-as-good-as-it-gets.html
[26] Alvarez, 90.
[27] Alvarez, 101.
[28] Kirsch, 80.
[29] Duggan EW, Thachenkary CS. (2004). Integrating nominal group technique and joint application development for improved systems requirements determination. Information & Management, 41, 399-411.
[30]
Jiang, 802.
[31] Browne and Ramesh, 631.
[32] Browne and Ramesh, 631.
[33] Alvarez, 86, 92.
[34] Analyzing
and defining requirements. (2013, September). Retrieved
from http://www.mitre.org/publications/systems-engineering-guide/se-lifecycle-building-blocks/requirements-engineering/analyzing-and-defining-requirements
[35] Tattersall G. (2015, October 27). Supporting iterative development through requirements management. Retrieved from http://www.ibm.com/developerworks/rational/library/2830.html
[36] Eliciting,
collecting, and developing requirements. (2015, August).
Retrieved from http://www.mitre.org/publications/systems-engineering-guide/se-lifecycle-building-blocks/requirements-engineering/eliciting-collecting-and-developing-requirements
[37] Duggan, 400.